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ABSTRACT 


The thesis investigates the determinants of effective and ineffective 
"instructorship" as perceived by students at the Naval Postgraduate School. 
Through a critical incident survey and subsequent content analysis a 
scheme which can express student perceptions of effective instructorship 
was developed. Categorization of good and bad instructorship incidents 
isolated sixteen factors which were found to be determinants of student 
perceptions. Examples of verbatim student comments are included in the 


form of a general feedback vehicle appropriate for instructor consideration. 
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I. INTRODUCTION 


As the educational goals of academic institutions become more demanding 
and time and cost considerations force accelerated programs of learning, 
increased attention has been focused on the quality of classroom instruc- 
tion. Attempts to measure the effectiveness of an instructor have placed 
a heavy emphasis on the solicitation of student opinion. The objective of 
the research reported here was to distill factors of instructorship which 
could express student perceptions of effectiveness. A survey of Naval Post- 
graduate School students conducted in January 19/73 provided the data base. 
This study furthers research initiated by Professors Elster, Githens and 


Senger at the Naval Postgraduate School in iigaoe 


pede it.opiidion regarding effectiveness can be useful both as a feedback 

aid to an instructor seeking to enhance his teaching and as a decision- 
making tool for administrative faculty review. This research has attempted 
to categorize student opinion into distinct common denominators of student 
perception regarding effectiveness or ineffectiveness. The primary thrust 
has been toward developing a coherent and Systematic scheme for addressing 
perceived instructor effectiveness determinants which could facilitate 
feedback. Quantification, in terms of rating scales, could eventually be 


based upon these common denominators. 





lR.s, Elster, W. H. Githens, and J. D. Senger, Factors Leading to 
Satisfaction and Dissatisfaction with Teachers, paper presented at the 
Western Psychological Association Convention, 1972. 





II. REVIEW OF LITERATURE 


A review of the literature relative to faculty evaluation reveals a 
diversity of methods utilized. Five of the most common procedures used 
in the evaluation of classroom teaching are discussed by Miller: class- 
room visitation by a qualified observer, routine monitoring of teaching 
materials and procedures, observation of special incidents relating to 
laudatory or negative aspects of an individual's teaching, self evaluation 
and student evaluation.“ A study conducted by Gaff and Wilson indicates 
that student rating is by far the most prevalent of the feasible procedures 
which have been used to cbtain systematic evidence about teaching effec- 
tiveness.> Miller however, notes that although some institutions of higher 
learning have experimented with student evaluation of teaching for several 
decades, numerous officials and professors have just discovered this 
dimension of evaluation and it has become the "in thing." 

A study of the many techniques utilized for student evaluation of 
teacher effectiveness acquaints the reader with a variety of evaluation 
forms which their respective authors believe to be the most effective. 
Few of these reports discuss the research involved in the development of 


the forms. The reader perceives that the vast majority of the devices 


are developed through an a priori or intuitive approach. The authors of 


2 Rp, I, Miller, Evaluating Faculty Performance (San Francisco: Jossey- 
Bass, 1965), pp. 26-39. 


3. Cc. Gaff, R. C. Wilson, and others, The Teaching Environment: A 
Study of Optimum Working Conditions for Effective College Teaching (Berkeley: 
Center for Research and Development in Higher Education, 1971), p. 290. 


4 Maller. Op, Cit., p. 14. 





these forms are professional educators and feel qualified to determine those 
qualities or factors which should be of primary importance in determining 
teacher effectiveness. The student evaluating the instructor is thus 
required to utilize a rating scale for each factor cited. As an example, 
Renner illustrates a form of this nature in his article, "A Successful 
Rating Scale." Through repeated use and periodic analysis of data 
collected, successive refinements and improvements can be implemented in 

the rating form. It must be noted that these refinements again assume an 
ability, on the part of the author, to determine what constitutes an improve- 
ment. A variation of this technique allows the student to report those 
additional factors (positive or negative) considered to be of importance 

by including a free-response section as part of the form. These responses 
might contribute to a revision of the form if the author finds in them a 
pattern in consonance with his views. Few rating forms have been oriented 
toward the student's perception of factors that constitute effective 
instruction. If student perceptions of instructor effectiveness are of 
value, logic would dictate that the factors which influence those perceptions 
should be identified. A definitive study of the factors Pose important to 
the student is not available. The primary focus of this research is to 

that end. 

The structure of the instructor evaluation forms used varies consider- 
ably although most use items that can be scored with relative ease. Items 
may be of the multiple-choice descriptive type or graphic type requiring 
the rater to check along a horizontal continuum. Sometimes the intervals 


in the graphic type are specific and carefully labeled while in other 





DR. R. Renner, "A Successful Rating Scale," Improving College and 
Wriversity Teaching, 15:12-14, 1967. 





cases the intervals are left vague. In either instance, the rater may 

be asked to circle a number or a letter grade or to put a check mark in a 
box. Frequently a five-point scale is used (with "5" representing "superior" 
and "1" representing "poor" or seed improvement"). Some items allow for 
only a dichotomous response: agree-disagree or yes-no. A few rating forms 
are entirely open-end, calling for short comments on such matters as 
instructor's attitudes toward his subject and/or attitude of the students 
toward him. The following illustrations are extracted from Kent's overview 


of these diverse devices routinely used for student evaluation of teachers.° 


A. MULTIPLE CHOICE DESCRIPTIVE 


Your opinion of the amount of homework for the course is: 
1. Superior, proper in amount and emphasis. 

2. Good, generally supplements work. 

au faar. 

4. Somewhat inadequate in value and proportion. 

5. Poor in most respects. 


B. GRAPHIC (SPECIFIC) 


Range of Interest and Culture 


10 9 8 fs 6 Ss) 4 3 2 1 0 
Instructor has very Instructor has Instructor is narrow 
broad interests and. fair breadth of in his interests and 
culture; frequently interests and culture; seldom 
relates course to - culture; occas- relates subject to 
other fields and to ionally relates other fields or to 
present-day problems. subject to other present-day problems. 


fields and to 
present-day 
problems. 


C. GRAPHIC (VAGUE) 


Does he appear sensitive to the students' feelings and problems? 





Unaware Responsive 





L. Kent, "Student Evaluation of Teaching,'' The Educational Record, 


Summer 1966, pp. 384-385. 
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D. 


FIVE POINT SCALE 


Circle the number most appropriate (5-superior, 1l-poor) 
Clarity of Speech : . Samp i453 Saez leeeeA 


PROJECTIVE FORM 


Complete the sentences to express your real feeling: 
Wiis BCOUTSE 64.0". 

mewiiat | frked .... 

I feel that the instructor... 

If I were teaching this course I would... 

The class discussions... 


i & WN Fe 


EXPLANATORY FORM 


The student should receive 50 minutes of instruction during each class 
period. An appropriate story, personal experience, or joke may illus- 
trate or emphasize a point while irrelevant stories and discussion 
unrelated to the subject waste time. How does the effective use of 
the period in this subject rank? 


The most striking impression obtained from reviewing the literature 


on evaluating teacher effectiveness is captured by Mitzel: 


“Although research has been in progress to this end for more than a half- 
century, the effort has not yielded meaningful, measurable criteria 
around which the majority of the nation's educators can rally. No 
Standards exist which are commonly agreed upon as the criteria of 

teacher effectiveness.''/ nd 





Tu. E. Mitzel, "Teacher Effectiveness," Encyclopedia of Educational 


Research (New York: The Macmillan Co., 1960) p. 148. 


Vt 





TIT. CRITICAL INGISENT METHOD 


For the purpose of neem a systematic approach toward resolving 
the determinants of effective classroom instruction we selected the method 
of critical incidents coupled with content analysis. The rationale for 
favoring this method was (1) an appreciation for the comparative objectivity 
of this approach and (2) the expectation of avoiding the major pitfalls 
of biased inferences that we suspected of some studies of teacher evaluation, 
Two basic approaches to content analysis are discussed by Herzberg 
et al..° The first approach is the a priori approach basing analysis 
upon a previously defined and outlined schematic system. An example of 
this approach would be the analysis of a body of material by sorting out 
factual from evaluative material with all of the material obtained falling 
into its appropriate predefined category. 
The second content analysis approach is an a posteriori method where 
the categories of analysis are extracted from the material itself. Our 
study adopted this latter approach to better enable a division into cate- 
gories that was more meaningful in terms of the empirical material 


gathered. 





Sr, Herzberg, B. Mausner, and others, The Motivation to Work (New York: 
John Wiley and Sons, 1967), p. 37. 
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IV. PROCEDURE 


From free written responses to a survey questionnaire we sought to 
construct categories of instructor characteristics which could express 


student perceptions of effectiveness and ineffectiveness. 


mm JHE SURVEY 

Students at the Naval Postgraduate School were asked to respond to 
two questions. The first asked them to think of a time when they were 
especially satisfied with an instructor and to Genera what led them to 
feel that way. The second question was identical, except that it concerned 


dissatisfaction (see Appendix A). Responses were received from 180 students 


from a distribution of 1496 questionnaires. 


B. ANALYSIS 

The second step was to sort and categorize the responses, through con- 
tent analysis, to determine mutually exclusive and nearly exhaustive 
factors leading to student satisfaction or dissatisfaction with instructors. 
This development of content categories became the central and most elusive 
aspect of the effort. On one hand, there was the danger of interpreting 
or superimposing preconceived ideas into the responses so that the structure 
and content of the categories could be made coherent. On the other hand, 
there was difficulty in classifying student perceptions of instructor 
behavior, attitudes, or efforts into categories which were really different 
from one another. 

i. Isolating Response Thoughts 


Before a response could be categorized, or grouped with others 


which were similar, a conceptual scheme was required to deal with the 
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language and processes of distinguishing oe thought expressed. It became 
useful to consider an opinion as an expression of a comparison between a 
situation as it was perceived and an idea about how it ought to have been. 
Thus the questionnaire prompted the sentiment of satisfaction or dissatis- 
faction and asked for the "situation-ought" comparison. This conceptual- 
ization became useful because it required that the three mutually dependent 
elements of opinion (1. perceived situation, 2. idea of ought, and, 
3. sentiment) be accounted for. We hypothesized that those situations 
which most frequently conformed to, or deviated from, an idea about what 
the situation should have been would be most frequently mentioned by the 
respondents. Similarly, this scheme suggests that a frequency tabulation 
of instructor effectiveness determinants will yield an order-of-importance 
listing which has significance only within the context of the situations 
in which the sentiments were evoked. The mutual dependence of the particu- 
lar situation and the idea of how it ought to be affects the relative fre- 
quency of response about any one factor. Each response is considered to 
describe a situation in conformity with, or in variance from, an idea 
about how it ought to be. Our data and analysis then, insofar as they 
reflect a priority listing of effectiveness determinants, apply to this 
particular sample. If the situations in the Naval Postgraduate School 
classrooms, Aer ee student ideas of what should be were different, the 
relative order-of-importance listing of Prrecrivetece determinants might 
also differ. 
2. Inference 

While clarification of the nature of opinion and its component 
elements helped conceptualize the task and acted as a brake to the tendency 
to “read into" the responses, a further problem was introduced by the 


various levels of inferences which were in fact contained in the responses. 
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On one hand, the students inferred or attributed attitudes, knowledge levels, 
and outside class effort to instructors. On the other hand, they sometimes 
inferred the reasons why the described attitudes, perceived knowledge 
levels, or attributed efforts — important to them. Inferences about 
the instructor were dealt with directly and incorporated into the cate- 
gorization scheme. Inference of the second sort, concerned with the 
reasons why an instructor's attribute was important to the student, consis- 
tently associated satisfaction with student fadetebes Sane When explicitly 
stated, competence gain or skill acquired comments were easily classified. 
A —- predominant theme was the implicit statement of, and concern for, 
comprehension which associated a behavioral or attitudinal characteristic 
of an instructor with understanding, (e.g., speaking clearly yielded 
satisfaction because it resulted in comprehension, or speaking in a clear 
manner yielded comprehension which resulted in satisfaction). In the 
cases where the expressed sentiment was associated with understanding and 
some other characteristics, (e.g., speaking clearly, legible Sénnne nee 
organized presentation, etc.,) the primary characteristic was recorded. 
The secondary or associational response concerning "understanding" was 
noted so frequently (752) that it was considered a basic common denominator 
an itself. 

3. Decision Tree 

Once the thought expressed in a response had been distinguished 

and the level or degree of inference present recognized, the categorization 
process required that it be explicitly identified. Repeated false 
Starts led us to adopt a consistent procedure for categorization. Rough 
categories were etched out and refined until a decision tree evolyed. In 
present form, the decision tree subjects each recorded thought of a respon- 


dent to a series of binary criteria which attempt to capture the thought. 
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The categorizer was required to make a binary yes/no decision regarding the 
match of each thought with each set of criteria in the categorization pro- 
cess. If a match could not be made the thought was rejected and subse- 


quently reviewed. 


C. VALIDATION 

A set of instructions was written documenting the decision tree pro- 
cedure for categorization (see Appendix B). The writers, working inde- 
pendently and following the documented instruction, obtained a 95% agreement 
on classification. In addition, similar agreement was obtained when a 
panel composed of the writers and three faculty members employed the 


procedure. 
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V. FINDINGS 


The flowchart or decision ee procedure for classification and cate- 
gorization of student perceptions clarifies the determinants of instructor 
effectiveness as perceived by students. Sixteen categories have been 
developed from student responses. The structure of the flowchart or 

- decision tree is simplified by five branchpoints which provide major 


decision criteria. 


A. RESPONDENTS’ UNDERLYING PREMISE 

Student concern for understanding the course material is the underlying 
theme throughout the responses. This concern is expressed explicitly as 
a determinant of satisfaction (how much was learned), directly as an 
attribute or category of instructor effectiveness (attitude toward student 
understanding), and indirectly associated with an attribute of the instructor 


(I learned because he taught at my level). 


B. BRANCHPOINTS : 

Descriptive titles or labels were assigned to each of the five branch- 
Meeats within the decision tree. These branchpoint labels serve to call 
to mind the criteria upon which a binary yes/no decision must be made. 
These are not categories in themselves, but are rather convenient ways of 
thinking. They are illustrated in Figure 1 and include: 

Instructor Knowledge 
Instructor Attitude Toward (Specific) 
Instructor Classroom Presentation Ability 


Instructor Effort Outside of Class 
Learning Result 


1. Instructor Knowledge 
Includes student opinion concerning instructor proficiency in 


subject matter. 
17 





FLOWCHART / DECISION TREE OUTLINE ILLUSTRATING BRANCHPOINTS 


ABLEIIY CLASS 
PRESENTATION 
a 






FIGURE 1 
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2. Instructor Attitude Toward 
Includes student perceptions of instructor attitudes toward student 
understanding, the subject matter, student questions, or the student in 
general. A "yes" decision at shale branchpoint is reached only if the 
attitude described is directed toward one of the above categories, e.¢., 
"He was enthusiastic" is excluded; "He was enthusiastic about the subject 
matter" is included. 
3. Instructor Classroom Presentation Ability 
Includes student's perceptions of instructor's abilities associated 
with presentation of material in the classroom. Emphasis here is on 
instructor activity characteristically associated with classroom presenta- 
tion, and does not include instructor activities in preparation for class- 
room presentation, evaluation of students, or other efforts normally made 
by the instructor outside of the classroom. 
4. Instructor Effort Outside of Class 
Included are student perceptions concerning instructor effort 
characteristically performed outside of the classroom. Additionally, all 
comments concerning instructor evaluation of students (e.g., examinations, 
grades, etc.) and course requirements are included. The “outside of class" 
or "effort" label is a convenient way of considering classification of a 
student perception and seems to be distinct from the actual presentation 
of material in the classroom--at least in the student's mind. 
>. Learning Result 
Includes only comments which specifically associate satisfaction 
or dissatisfaction with student competence or skill gain. Excluded are 
comments which associate understanding or learning with an attitude, 


ability, or effort of the instructor. 
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C. CATEGORIES 
Aithoush each of the sixteen categories within the decision tree are 
subordinated to a branchpoint, they are meant to stand alone, outside of 
the context of the branchpoints which were used as a convenience when 
categorizing. The complete flowchart/decision tree is illustrated in 
Figure 2. Categories are discussed below in order of significance or 
frequency found in the survey. 
1. Organization of the Course 
This category encompasses the orientation of the course toward 
clear objectives. Included are the perceived clarity of course require- 
ments, studies to reach the objectives and relevance of the course to 
larger programs of study or "real world" situations. Separation of important 
from less-important material and a balance between theory and the practical 
were important factors. In this category the direction of the course as 
a whole, in terms of how it actually proceeds, is central, e.g., ''We 
always drifted from topic to topic, without any idea of where we were 
going or why." 
2. Evaluation of Students 
Every aspect of grade assignment is included in this category. Its 
position as second most Eyes mentioned is dependent upon incorporating 
all aspects into one category. To distinguish the significance of compo- 
nent factors included, subcategories were constructed from those responses 
which indicated "evaluation of students" as a factor in perceived effec- 
tiveness. The subcategories included: 
Content of examinations 
Fairness in grading examinations 
"Taxingness" of assignments 
Grading scheme or pressure on grades 


Promptness in returning examinations 
Actually reading assignments submitted 
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These subcategories were then ranked in order of frequency mentioned with 
the other categories. With one exception, each subcategory ranked below 


\ 


all other factors of satisfaction or dissatisfaction. The exception, 


' ranked tenth among all other categories and 


"content of examinations,' 
seemed to be the predominant issue concerning evaluation of students. 

The negative correlation between examination content and material covered 
in the course was frequently mentioned by students, e.g., "The content 

of ae examinations did not resemble the material we had .covered in class." 


3. Instructor Attitude Toward Student Understanding 


This category reflects instructor concern, interest, desire, etc. 
for student comprehension or progress. One extreme is reflected in the 
comment, "He doesn't just cover topics, he wants us to learn the material." 

4. Instructor Ability to Create a Learning Environment and Demeanor 

This category captured the elusive qualities of enthusiasm, per- 
sonality, honesty, open-mindedness as well as abilities to inspire or 
motivate. The authors suspect that the ability to be appealing is probably 
the most difficult to address or influence, especially through student 


te 


feedback. 
oe Prior Preparation 
“Student perceptions regarding the degree of prior preparation for 
a classroom pefiod are included in this category. 
6. Clarity of Explanation 
Comments concerning an instructor's ability to clearly explain were 
categorized here. Vivid use of examples, clarity when presenting difficult 
Material, and direct responses to questions were frequently noted responses, 


e.g., “His clarity (of presentation) made the difficult material as easy 


as possible." 


22 








). Instructor Knowledge 


This category includes depth and breadth of instructor proficiency 
in the course material. A frequent negative comment was, "He couldn't 
work homework problems on the board." 

8. Instructor Ability to Teach at Appropriate Level 

This category includes comments concerning instructor awareness 
of student level of comprehension, and adjustment of the pace and level 
of instruction appropriate to student ability and prior knowledge, e.g., 
"He taught over our heads and was unaware that no one understood what 
Was=going on." 

9. Instructor Attitude Toward Course and Subject Matter 

Included are comments concerning instructor interest in and respect 

for the subject matter. Opposites in this category are "He was interested 


<9 


in the field" and "He said the ccurse was useless." 


10. Organization of Lecture Material 
The structure, logic, and sequence of material presented in class 
is included in this category. Summarizing, providing an overview, empha- 
sizing the significant in a logical sequence, and "being flexible, without 
getting lost"’ are frequently-noted positive characteristics. 
m6 -Instructor het teudeeroward the Student 
This category refers to comments concerning instructor-student 
interrelationships in which the student discerns an attitude specifically 
toward him as a person (e.g., aloofness, patience, encouragement). Not 
included are either more specific instructor attitudes toward student 
understanding and questions or more general personality traits (honesty) 
or characteristics ("appealingness"). 
12. Student Competence Gain 


Only direct comments which associate satisfaction or dissatisfaction 
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with amount learned are included. "I was satisfied because I learned a 
lot" or "I was dissatisfied because I didn't learn anything and the whole 
thing was a waste of time.'’ As previously noted, student concern for their 
understanding was implicit in a large majority of the responses. 
13. Instructor Attitude Toward Student Questions 
Expressions of instructor willingness to listen to and answer 
questions in addition to student perceptions of instructor concern for the 
content of student understanding are included in this category. Typically, 
"He wants students to understand concepts not just mechanics," and "Never- 
mind-how-it-works-just-plug-in-the-numbers type attitude." 
14, Instructor Ability to Control Class and Time 
Comments concerning effective use of class time and class partici- 
pation are categorized here. 
15. Instructor Availability 
Comments concerning outside class availability to lend assistance 
to students are included, e.g., "He was always available for outside 
assistance." 
fone Instructor Ability.to Evoke Interest 
The instructor's ability to evoke interest is contrasted to, "He 


reads from the text," in this category. 


D. INTERPRETATION AND TABULATION 

Our findings suggest that the defined categories and procedural method 
of categorization can express student perceptions of instructor effective- 
ness in the vocabularies and within the frame of reference of students at 
the Naval Postgraduate School. The rankings of the categories in order 
of importance reflects a moment in time and a particular sample. It is 


our hypothesis that at a different moment of time or by using a different 
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sample the categories would stand while their order of ranking (according 
to their frequency) might be different. Table I presents the tabulated 
frequency of responses found-in the January 1973 sample. In our estimation 
it would be erroneous to conclude from this tabulation that students are 
more satisfied than dissatisfied with instruction on the basis of positive 
or negative responses to the summation of particular categories. Similarly, 
it does not appear as if any of the categories represent satisfaction signif- 
icantly more than dissatisfaction, or vice versa. On the other hand, it 
may be that instructor emphasis on categories toward the higher end of the 
ranking could more quickly and effectively lead to improved effectiveness 
as perceived by students. 

Another perspective of the ranking order of categories can be gained 
if they are considered in light of either prescriptions or proscriptions 
for action. In this case two observations are suggested. 

1. Evaluation of Students 

If student evaluation or "grades" play such a dominant part in 

student perception, one questions what it is, in particular, that plays 
that vital part. Additionally, one questions how those factors related 
to student evaluation compare with the other categories. These findings 
indicate, as previously aageup that content of examinations is the central 
issue, Rearrangement of the frequency tabulation to account for the sub- 
categories or influences detected within student evaluation yields the 
rankings in Table 2. Evaluation of students in Table 2 is broken into 
Six Mibcatecories and ranked with the other fifteen categories. Exami- 
nation content then ranks tenth, while other facets of student evaluation 


are found in positions 17 through 21. 
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ORDER-OF-IMPORTANCE CATEGORY LISTING AND FREQUENCY TABULATION 
‘TABLE I 


Rank Category Total # # of Associations # of Associations 
Order Description Mentioned with Satisfaction with Dissatisfaction 


1 Course Organization 153 wy 76 
2 Evaluation of Student 9 60 79 

3 Attitude Toward 
Student Understanding 112 63 49 

4 Ability to Create 
Learning Environment ied 60 51 
5 Instructor Preparation 106 46 60 
6 Boulity to Explain 100 60 40 
7 Instructor Knowledge 97 52 45 
8 Ability to Teach at 
Appropriate Level 72 41 31 
, Instructor Attitude 

Toward Course 68 40 28 
10 Lecture Organization 62 28 34 
a). Attitude Toward Student 58 34 24 

12 Attitude Toward- 
Questions 49 24 25 
13 Student Learning Result 35 19 16 
14 ete Control 34 PZ 22 
pS, Instructor Availability 30 23 7 
16 Ability to Evoke Interest 29 14 15 
TOTALS i255 653 602 
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2. Distinctions 

The development of the categories was an attempt to express 
@eterminants of instructor effectiveness as perceived by students. The 
effort has sought to make distinctions between the categories clear, 
distinct and mutually exclusive. The method has not been able to satis-— 
factorily address the degree to which the categories are statistically 
different. The constructs of language and multiple phrases which can 
express the same thought may have led us to construct categories which 
seem to be different, but which may not be. The overwhelming emphasis on 
"organization" is the predominant instance. 

Comments concerning organization seemed to refer to either the 
course in general or to the lecture presentation. Organization of lectures 
was sometimes attributed to preparation, in which case it was categorized 
separately. Classifying course organization, organization of lectures, 
and preparation into one category would have been, in our minds, an 
inference. Although instructors themselves may equate the prior preparation 
of a lecture with an organized presentation, the categories have not been 
constructed as if the same cause-effect correlation is ee by the student. 
The language of our branchpoints (instructor effort and instructor ability) 
kept this distinction. Thus, comments to be classified under preparation 
were recognized by the word "prepare" whereas comments categorized under 
lecture organization referred to either the results of that preparation 
before ("His lectures were organized") or after ("His lectures were pre- 
sented in an organized fashion") presentation. If these three categories 
are considered one, comments concerning organization account for 25% (312) 


of the responses. 


fay 





E. COMPARISON WITH A SIMILAR STUDY 
As this study drew to a close, a study by W. W. Ronan reporting a 


2 


similar critical incident study became available. In comparison, the 
results of that study delineated seven subdivisions or types of instructor 
behavior (roughly analogous to our six branchpoints) and ninety-seven 
specific behaviors or actions associated with satisfaction or dissatis- 
faction (roughly analogous to our sixteen categories). The essential 
difference between the results reported herein and those -found in the 


Ronan study is our concentration on student perceptions versus an 


exclusive concentration on specific instructor actions. 





Se Department of H. E. W./Office of Education, PREP Report No. 34, 
Evaluating College Classroom Teaching Effectiveness, by W. W. Ronan. 
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ORDER-OF-IMPORTANCE CATEGORY LISTING AND FREQUENCY TABULATION 
SUBCATEGORIZING FACTORS IN STUDENT EVALUATION 


TAB Ee 2 
Rank Category Total # # of Associations # of Associations 
Order Description Mentioned with Satisfaction with Dissatisfaction 
1 Course Organization 153 77 76 
2 Attitude Toward 
Student Understanding 112 63 49 
3 Abddldshyaeto oCreate 
Learning Environment aL 60 51 
4 Instructor Preparation 106 46 60 
5 Ability to Explain 100 60 40 
6 Instructor Knowledge oy 52 45 
7 Ability to Teach at 
Appropriate Level 72 41 Sul 
8 ime trugtor Attitude 
Toward Course 68 40 28 
9 Lecture Organization 62 28 34 
10 Examination Content 61 Z2 She 
11 Attitude Toward Student 58 34 24 
12 Attitude Toward Questions 49 24 25 
13 Student Learning Result 35 IY 16 
14 _ Instructor Control 34 12 22 
15 Instructor Availability 30 23 7 
16 Ability to Evoke Interest 29 14 15 
7 Fairness in Grading 
Examinations 28 12 16 
18 Taxingness of Course Pad IL) 11 
19 Grading Scheme 17 10 7 
20 Promptness Returning 
Examinations 11 5 6 
21 Attention to Submitted Work 2 1 1 
TOTALS 1255 653 602 
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VL. seABBIeLCATIONS 


Two applications of the findings to enhance feedback to instructors 


are suggested. 


A. GENERAL FEEDBACK 

The development of a document which could make available the comments 
of students recorded during the survey in the format of the developed cate- 
eories might prove of some value to instructors seeking to enhance their 
effectiveness. To this end a general description of our efforts and ver- 
batim comments classified according to developed categories has been com- 
posed and is included as Appendix C. In addition, this appendix provides 


documentation for the classification procedure described earlier. 


B. SPECIFIC ROUTINE FEEDBACK 

A vehicle and a procedure which could be routinely employed to gather 
student perceptions or opinions of instructor effectiveness could be 
based on the categories distilled from this effort. Development of 


Suitable scales awaits future efforts. 
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APPENDIX A 


SURVEY QUESTIONNALRE 


whe instructor Ba tise or break the course. Student opinion too often 
travels via the grapevine of acquaintances instead of back to the instructor 
or to those making administrative decisions. If student evaluation is to 
be significant, the determinants of student evaluation must be clarified. 
This questionnaire seeks to clarify what should be evaluated or measured 
about an instructor's performance from the perspective of the NPS student. 

Your comments on page two will assist in this endeavor. Identification 
of instructors by name is specifically NOT solicited. Please leave the 
completed questionnaire in the "instructor questionnaire" box in the 
Student Mail Center. 

This questionnaire is distributed IAW NPS INST. 1520.13 by students 
fulfilling the requirements for MN 0810, Thesis Research, under the 
direction of Professor R. S. Elster. 


Thank you for your assistance. 


Sul 





Curriculum Qtrs Completed at NPS 


QPR Overall Yrs Military Service 


Think of a time when you were particularly satisfied with an instructor. 
What led up to this feeling? 


Think of a time when you were not satisfied with an instructor. What 
led up to this feeling? 
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APPENDIX B 
INSTRUCTIONS FOR CATEGORIZING STUDENT RESPONSES TO 


CRITICAL INCIDENT QUESTIONNAIRES CONCERNING INSTRUCTION EFFECTIVENESS 


I. INTRODUCTION 


Questionnaires soliciting student opinions concerning the factors of 
instructor effectiveness at the Naval Postgraduate School require processing. 
Each returned questionnaire contains free-written responses which indicate 
factors important to the respondent. The task is to identify like factors 
and group them into categories. 

The categorization process involves two steps: first, distinguishing 
the thoughts within the response, and second, categorizing the distinguished 
thoughts into categories containing similar thoughts. Certain concepts 


and procedures assist in this process. 


II. ISOLATING THE THOUGHT 


Conceptual clarification of the nature of the student responses can 
be helpful. Students have recorded their opinions regarding positive and 
negative aspects of instructor effectiveness. These opinions consist of 
three elements, Sone or all of which may be present in any given response. 
When seeking an opinion, a response indicating like or dislike, in itself, 
is considered to be a comparison between the situation called to mind and 
the sentiment evoked. If the reason for the sentiment is explained, that 
explanation or elaboration makes a comparison between the situation as it 
1s perceived and an idea about how it should have been. To a large extent, 


the responses, and hence the categories into which they are classified, 


® 


oe 





represent variances from, or conformity with, ideas about “how things 
should be." 

Each categorizer must consciously avoid reading his own thoughts or 
past experiences into responses. The responses must be eae at their 
face value and not interpreted to represent thoughts consistent with the 
experience or bias of the categorizer. This is more easily said than done. 
It is the individual thoughts that must be categorized, not the general 


gist of the response. 


III. CATEGORIZATION OF THE THOUGHT 


The categorization procedure involves processing each distinguished 
thought (which may be a phrase, a sentence, or a paragraph) through a 
flowchart which has five branchpoints and sixteen categories. At each 
branchpoint and each category a binary yes/no decision is required to 
determine the next path or appropriate action. Each decision is based 
on criteria within the decision box. If the criteria within the decision 
box “capture'' the thought being processed, follow the "yes" path. If not, 
follow the "no" path. Each thought is allowed one pass through the flow- 
chart; if uncaptured it ends in a reject box and is logged accordingly. 
Branchpoints have the one-digit numbers one through five. Categories 
have two digit numbers. Each category is subordinate to one branchpoint, 
although two of the five branchpoints have only one category (1 has 11, 
amas 5 has 51). 

Each branchpoint and category is described by a set of criteria upon 
which the yes/no decision is based. In addition to the criteria and a 
number, each also has a word or name which serves the function of calling 


the criteria to mind. 
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The branchpoints are not categories; they merely assist in making 
the correct decision based on the criteria they contain. Again, the 
names serve to call the criteria to mind. The branchpoints are numbered 
and labeled as: 

Instructor Knowledge 
Instructor Attitude Toward 
Instructor Classroom Presentation Ability 


Instructor Effort Outside Class 
Student Learning Result 


i WN Fe 


A condensed version of the flowchart is presented in Figure A-l. 

Each thought is processed through the flowchart until it is either 
captured by the criteria of a category or is rejected. Once captured, 
the corresponding number of the category is logged on a three-by-five 
inch card sequentially numbered to correspond with each questionnaire. 
Figures A-2 and A-3 illustrate the format. The category numbers of 
thoughts associated with satisfaction (the top half of the questionnaire) 
are logged on the left half of the card; those associated with dissatis- 
faction (the bottom half), on the right. If a thought is rejected, the 
response is logged verbatim on the reverse side of the card. 

Figure A-4 displays the entire flowchart/decision eres and includes 
category labels. The addendum includes the criteria which these labels 
represent and examples of thoughts included and excluded in each category. 
As a procedural method, the addendum should be disassembled and organized 
into a single chart. Figure A-4 will then serve as a condensed mapping 
reference. Prior familiarity with the criteria is essential and organi- 
zation into a single diagram facilitates categorization. 

To summarize, the categorizer has before him sequentially numbered 
Survey questionnaires, correspondingly numbered cards, a schematic repre- 
sentation of the flowchart, and Figure A-4. Thoughts expressed in the 


questionnaires are processed through the flowchart. The number of the 
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FLOWCHART / DECISION TREE OUTLINE ILLUSTRATING BRANCHPOINTS 





FIGURE A-1 
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Category numbers 
corresponding to 
the thoughts ex- 
pressed in the 

top half of this 


questionnaire. 


Category numbers Sequential 
corresponding to number 

the thoughts ex- 

pressed in the 

lower half of 

this questionnaire. 





Figure A-2 





Figure A-3 
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category capturing each thought is logged on the appropriate half of the 
card. If a thought is rejected, it is recorded on the back side of the 


fard. 


LV. CONCEPTUAL AIDS 


Initial difficulty recording comments concerning "organization" and 
"student understanding’ can be anticipated. Distinctions between comments 
addressing either of these subjects follow specific rules within the flow- 


emart. 


A. STUDENT UNDERSTANDING 

Underlying many of the responses is the implicit expression of the 
student's idea that he ought to learn and that the instructor ought to 
teach in a manner such that learning results. These kinds of thoughts 
are handled in two ways. If the thought directly, and without qualification 
or association with an instructor attribute, expresses satisfaction or 
dissatisfaction on the basis of student competence gain, it is categorized 
under "Learning Result" (51). For example, "I was satisfied because I 
learned the subject material" or "The course was a complete waste of time 


and I didn't learn a thing." 


On the other hand, if understanding is 
addressed in the context of, or is associated with, an instructor attribute, 
the thought is logged according to the attribute with which it is associated. 


For example, ''He was concerned with my understanding" (21) or "His explana- 


tions were clear and I could understand them" (32). 


B. ORGANIZATION 
A second predominant theme in the responses concerns organization. 


The decision tree distinguishes between three closely related concepts. 
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The first distinction rests between the organization of the course (44) 
and the organization of the lecture (34). While the distinction seems 
clear enough, course organization is a broader concept and includes 
general comments about organization (e.g., "He was disorganized.'"). More 
specific comments may refer to either the course as a whole or the class- 
room presentation of the material. A second distinction rests between 
preparation (41) and lecture organization (34). The category including 
preparation is almost always recognized by the word "prepare.'' Lecture 
organization, on the other hand, denotes both the results of preparation 
("His lectures were organized") and the results of presentation ("He 
presented the material in an peeeneea manner.")., Although it seems 
logical to infer that preparation results in an organized lecture presenta- 
tion, it is an inference nevertheless. To the greatest extent possible 
the categorization procedure seeks to avoid inferences on the part of the 


Categorizer. 


V. LOGGING RULES 


= 


When recording the numbers of categorizers which have captured thoughts, 
any given category number may be recorded only once for the positive and 
once for the negative expression in any one response. Thus, if the same 
thought is expressed in several sentences it is recorded only once vice 
once for each sentence. In those instances when the respondent has 
indicated "the converse of the above" on the bottom half of the question- 
naire, each category number recorded on the left side of the card is also 
logged on the right. Several practice runs through the procedure, adhering 


to the yes/no decision at each point in the flowchart, are recommended. 
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ADDENDUM I TO APPENDIX B 


FLOWCHART BRANCHPOINT AND CATEGORY DECISION 


CRITERIA 
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PAGE 1 


DLART HERE 


INSTRUCTION 
Go to page 2 
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Entry from page l 
PAGE 2 
Branchpoint label: KNOWLEDGE (1) 
Subordinate Categories: MSPictoy Knowledge (11) 


Criteria: Instructor knowledge of the subject he is teaching 
Knowledge in his field 
Includes depth and breadth of knowledge 
Demonstrated proficiency in subject matter 


Examples: "He was knowledgeable in his field." 
"He couldn't work homework problems on the board." 
"He was unable to relate the subject matter to the real world." 


This category does not include statements such as 
"He did not relate the subject matter to the real world." or 
"He did not work homework problems on the board." 


INSTRUCTION/DECISION 
"YES vomro pace 25 
"NO" go to page 4 
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Entry from page 2 
PAGE 3 
Category: INSTRUCTOR KNOWLEDGE (11) 
Superior branchpoint: KNOWLEDGE (1) 


Criteria: The criteria for this category are the same as the criteria for 
the branchpoint. Has the thought been captured? 


INSTRUCTION/ DECISION 
"YES" log category 1l 
MNO" reject 
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Entry from page 2 
PAGE 4 


Branchpoint label: ATTITUDE TOWARD (2) 


Subordinate categories: Student understanding/teaching (21) 
Subject matter/field (22) 
Student Questions (23) 
Students (24) 


Criteria: Attitudes, but must be directed toward one of the subordinate 
categories. 
Words which help identify attitudes include enthusiasm for, 
interest in, concern for, willingness to, involvement in, 
desire to, and wants. 


Examples: "He was encouraging" YES 
"He was enthusiastic" NO 
"He was enthusiastic about the subject" YES 
"He really wants to teach" YES 
"He was aware of .. ."' NO 
"He adjusted the pace" NO 
"He adjusted the pace because he really wanted us to understand" YES . 
"He cares about what we learn" YES 


INSTRUCTION/DECISION 
"YES" go to page 5 
"NO" go to page 9 
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Entry from page 4 
PAGE 5 
Category: STUDENT UNDERSTANDING (2A) 
Superior branchpoint: Attitude — (2) 
Criteria: Attitude toward student progress 
Attitude toward teaching 


Attitude toward student comprehension of material 


Examples: "He doesn't just cover topics, he wants us to learn the material." 


INSTRUCTION/DECISION 
“YES” log category 21 
"NO" go to page 6 
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Entry from page 5 


Category: SUBJECT MATTER/FIELD (22) 
Superior branchpoint: Attitude toward (2) 


Criteria: Interest in the field 
Respect for the field 
Attitude toward course material 


Examples: ''He said the course was useless." 
“He was interested in the field." 


INSTRUCTION/DECISION 
"YES" log category 22 


"NO" go to page 7 
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PAGE 6 





Entry from page 6 
PAGE 7 
Category: STUDENT QUESTIONS (23) 
Superior branchpoint: Attitude — (2) 


Criteria: Willingness to listen to and answer student questions 
Concern for the content of student understanding 


Examples: “He is offended by questions." 
"He wants students to understand concepts not just mechanics." 
"Never—-mind-how~it-works-just-plug-in-the-numbers type attitude." 


Not included in this category are comments such as 
"He teaches concepts not just mechanics." 


INSTRUCTION/DECISION 
myES lop catecoryecs 
"NO" go to page 8 
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Category: 


' Entry from page -7 
PAGE 8 


STUDENT (24) 


Superior branchpoint: Attitude toward (2) 


ereeria: 


Examples: 


His manner in relation to students, but not his manner in general. 
Implies a specific trait in relation to students. 


"He is aloof." 

"He is encouraging." 

"He is patient." 

"He has a superior attitude." 


But mot , 

"He has a good personality." 
"He has colorful speech." 
"He is a good showman." 

"He is enthusiastic." 

"He is honest." 

"He was appealing." 


Comments such as "He was inspiring" do not belong in this category. 
“He made me want to learn" is thought of as an ability vice an 
attitude of the instructor and is subordinated to branchpoint 3. 


INSTRUCTION/DECISION 
"YES" log category 24 
MNO" 6 reject 
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Entry from page 4 


PAGE 9 


Branchpoint label: CLASSROOM PRESENTATION ABILITY (3) 


Subordinate Categories: To teach at appropriate level (31) 


Griteria: 


Examples: 


To explain (32) 

To evoke interest (33) 

To organize lecture (34) 

To control time/participation (35) 

To create a learning environment and demeanor (36) 


Characteristic of material presentation in the classroom during 
a class period 
Abilities or attributes 


"He adjusted the pace." 

"He read from the text." 

“He was enthusta@stic. 

"His board work was neat and legible." 

"He is aware of the degree of class understanding." 
"He puts material across." 

"He made me want to learn." 

"He was narrowminded." 

"He speaks well." 

eHe iss colorful.“ 

"His lectures were presented in an organized manner." 
"His lectures were organized." 


Not included in this category are comments reflecting preparing 
for lectures, organizing the course, and evaluating the student. 
Although the student may observe these in the classroom they 
are thought of as characteristic activities of the instructor 
outside of class. Examples are "He was organized." (see 44) 
and "He was prepared." (see 41). 


INSTRUCTION/DECISION 
"YES' go to page 10 
"NO" go to page 16 
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Entry from page 9 
PAGE 10 

Category: TO TEACH AT APPROPRIATE LEVEL (31) 
Superior branchpoint: Classroom presentation ability (3) 
Criteria: Teach at the level of student knowledge 

Teach at the level of student ability 

To be aware of student comprehension 
Examples: ''He used layman's terms." 


"He could adjust the pace." 
"Just right as far as challenge goes." 


INSTRUCTION/DECISION 
mins: doe Category 31: 
"NO" go to page 11 
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Entry from page 10 


PAGE 11 


Category: TO EXPLAIN (32) 


Superior branchpoint: Classroom presentation ability (3) 


Criteria: Clarity of explanation 
Good imparter of knowledge 
Ability to answer questions 
To use examples to clarify 
To make material understandable 


INSTRUCTION/DECISION 
"YES" log category 32 
"NO" go to page 12 
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Entry from page il 
PAGE 12 
Category: TO BE INTERESTING (33) 
Superior branchpoint: cites test escentaeien ability (3) 


Criteria: To evoke interest 
To make material interesting 


Examples: The most frequent negative comment in this category is 
"He reads from the text." 


INSTRUCTION/DECISION 
"YES" log category 33 
Unie comtompagem):3 


aS 








Fntry from page’ 12 
PAGE 13 


Category: TO ORGANIZE LECTURE MATERIAL (34) 
TO PRESENT MATERIAL IN AN ORGANIZED MANNER 


Superior branchpoint: Classroom presentation ability (3) 


Criteria: Structure, sequence, and "logicalness" of material presented 
during a lecture 
Pointing to significant points in a lecture 
Summarizing, overviewing, emphasizing 
Being flexible, without getting lost 


Examples: Does not include organization or composition of the course 
Not to be confused with preparedness, which may or may not 
result in an organized presentation 
Not to be confused with more general comments about organization 
such as "He was organized," 


INSTRUCTION/DECISION 
"YES" log category 34 
"NO" go to page 14 
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Entry from page 13 
PAGE 14 
Category: TO CONTROL TIME/PARTICIPATION (35) 
Superior branchpoint: Classroom —— ability (3) 
Criteria: Effective use of class participation 


Effective use of class time 
Control of direction and content of classroom instruction 


INSTRUCT ION/DECISION 
iWEe lor catesory 53 
"NO" so to page 15 
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Entry from page 14 
PAGE 15 
Category: TO CREATE LEARNING ENVIRONMENT AND DEMEANOR (36) 
Superior branchpoint: Classroom presentation ability (3) 


Criteria: To inspire, motivate 
To exemplify appealing attributes, traits, characteristics 
To present a learning demeanor 
To create a learning environment 


Examples: "Showmanship" 
"Personality" 
"Enthusiasm" 
"Honesty" 
"Blackboard techniques" 
"He made me want to learn." 
"He talks too much." 
"He always tries to prove how smart he is." 
"Narrowminded" 
Rapport 


INSTRUCTION/DECISION 
"YES" log category 36 
“NO” revect 
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Entry from page 9 


PAGE 16 


Branchpoint label: EFFORT OUTSIDE CLASS (4) 


Subordinate categories: To prepare (41) 


Criteria: 


TO be avalleble 42) 
To evaluate students (43) 
To organize course (44) 


Instructor responsibilities characteristically performed outside 
of the classroom 

All references to grades, examinations, evaluation of students, 
and "taxingness" of work 

Relevance and clarity of course objectives and requirements 
Emphasis and relevance of course material 

General remarks concerning organization 

Preparation of material for classroom presentation 


INSTRUCTION/DECISION 
tYES. eo EO pace ol/ 
"NO" po to pape 21 
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Entry from page 16 
PAGE 17 
Category: TO PREPARE (41) 
Superior branchpoint: Effort sass class (4) 
Criteria: Instructor preparedness for lectures 


Examples: Usually recognized by word "prepare" 


INSTRUCTION /DECISION 
"YES'' log category 41 
"NO" go to page 18 
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Emtry from page 17 
PAGE 18 
Category: TO BE AVAILABLE (42) 
Superior branchpoint: Effort eubeide ellass (4) 
Criteria: Offering outside assistance 


Being available for help 
Readiness to assist 


INSTRUCTION/DECISION 
MYES” log Category 42 
"NO" go to page 19 
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Entry from page 18 
PAGE 19 


Category: TO EVALUATE STUDENTS (43) 
Superior branchpoint: Effort outside class (4) 


Criteria: Content and composition of examinations 
Fairness grading examinations 
"Taxingness'' of work 
Grading scheme, including emphasis or pressure on grades 
Reading submitted assignments 
Promptness in returning examinations 


INSTRUCTION/ DECISION 
"YES" log category 43 
"NO" go to page 20 


60 









Entry from page 19 
PAGE 20 
Category: TO ORGANIZE COURSE (44) 
Superior branchpoint: Effort Sues le class (4) 


Criteria: Cohesiveness of course 
Clarity of course objectives and requirements 
separation of important material from less important 
Orientation of course: Theory versus Practical 
Relevance to real world 
Relevance of assignments to objectives 
General comments concerning organization 
Tendency of course material to be a unit 


INSTRUCTION/DECISION 
"YES" log category 44 
"NO" reject 
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Entry from page 16 
PAGE 21 
Branchpoint label: LEARNING RESULT (5) 
Subordinate category: Sees gain (51) 


Criteria: Explicit statement of satisfaction as a result of the amount 
learned, skill acquired, competence gained, a learning experi- 
ence, or understanding obtained. 

Does not include comments which associate resultant understanding 
with another instructor attribute. 


INSTRUCTION/DECISION 
MYES "go to page 22 
"NO" reject 
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Entry from page 21 
PAGE 22 
Category: COMPETENCE GAIN (51) 
Superior branchpoint: Learning result (5) 


Criteria: The criteria for this category are the same as those for the 
branchpoint. Has the thought been captured? 


INSTRUCTION/DECISION 
"YES" log category 51 
"NO" reject 
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APPENDIX C 


STUDENT PERCEPTIONS. OF INSTRUCTOR EFFECTIVENESS 


Different openings for this discussion of students' perception of 
teachers have been pondered, like '" .. . what you've always wanted to 
know about students' perceptions but were afraid to ask..." or, io sre 


just one hour toward a more tantalizing teaching image . . ." or, perhaps, 


" . . . how to be one up on your colleagues on student opinion of teaching 


questionnaires at NPS .. .."' However, any of these statements might be 
considered rather cheeky. It seems better to dispense with the statements 


‘on the facts;"’ here they are. 


and the discussion, and bore in 
In an attempt to clarify the factors of instructor effectiveness as 
perceived by students a survey was conducted in January 1973. All Naval 


Postgraduate School students were asked to respond to two questions: 


1. Think of a time when you were particularly satisfied with an instruc- 
tor. What led up to this feeling? 


2. Think of a time when you were not satisfied with an instructor. What 
led up to this feeling? 


Student responses were categorized according to content in an effort to 
group similar answers. Once these categories were developed, it was 
envisioned that a vehicle could be developed in which routine feedback to 
instructors could be provided. Secondly, a more general feedback could be 
obtained if the kinds of responses received to the questionnaire in January 
were made available. These pages seek to make that general kind of feed- 
back available. 

The effort to group responses such that a large number of student 
perceptions could be coherently expressed resulted in sixteen categories. 


Although the development of these categories was the major effort associated 
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with the survey, the verbatim student responses to the questionnaire are 
provided here to illustrate the kind of comments received. Several pre- 


dominant themes seem to underlie these responses. 


® 


i nether implicitly or explicitly stated, student understanding of 

course material, or facilitation of student comprehension seemed to be the 
bases upon which satisfaction or dissatisfaction were expressed. Particular 
attitudes or actions on the part of the instructor were most frequently 
singled out because of the contribution they were felt es make to student 


understanding. 


B. There seemed to be an overwhelming student emphasis on organization of 
the course and classroom presentation. This emphasis was manifested in 
comments concerning organization of the course, organization of the class- 
room presentation, and prior preparation, which, for purposes of this study, 
were categorized separately. Student experience in a military environment 


may play some part in this emphasis. 


C. Each of the categories was associated about as frequently with student 


satisfaction as with student dissatisfaction. 


i onssmight be expected, SeRMERES associating satisfaction or dissatisfaction 
with the evaluation of students by instructors were frequently noted. Further 
subdivision of this category of responses indicated that a main issue in 

the students' minds was the content of examinations, i.e., appropriateness 


Ommreaterial, difficulty, length, etc.. 


E. The categories and verbatim responses that follow are presented in 


Grder of their frequency of mention. 
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Course organization 

Instructor evaluation of student 

Instructor attitude toward student understanding 
Ability to create learning environment/demeanor 
Instruction preparation 

Clarity of explanation 

Instructor knowledge 

Ability to teach at appropriate level 
Instructor attitude toward course 

Lecture organization 

Instructor attitude toward student 

Instructor attitude toward questions 

Student competence gain 

Imstructor control of class time 

Instructor availability 

Ability to evoke interest 


While the categories might be expected to reflect the factors students 


perceive as being associated with effectiveness or ineffectiveness of 


teaching in any sample, one would not necessarily expect the ranking to 


remain the same in another sample, say at another school, or, at another 


time. 


For purposes of illustration, the context of student remarks has 


often been included below even thought the surrounding context might have 


otherwise fit into another category during the classification procedure. 


Verbatim student comments follow. 
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ORGANIZATION OF THE COURSE 


Courses where the student must grope for what the course is to cover 
because the instructor has failed to structure an outline to conform 
with the catalogue; i.e., the instructor seems to lecture on whatever 
strikes his fancy, no course outline prepared, avoids questions related 
Be course content. 


Provided the student with an overview of what he was trying to teach, 
its place in the overall discipline, the importance or reasons for 
studying it, and its applications. 


He clearly defined the course and what was expected in the way of home- 
work, tests, papers, etc., from the outset. 


An outline of the course was presented, at the beginning, with things 
that resembled course objectives. The material seemed pertinent to 
our total field of study. Reasonable assignments were given that 
reinforced and complimented classroom work. 


Disorganization--skipping around from section to section in a text during 
the lecture and then from lecture to lecture, without stating where 
lecture material is coming from and without providing daily homework 
assignments to either supplement the previous day's lecture or prepare 
for the next one. 


The material was taught from a practical point of view rather than a 
series of mental gymnastic exercises. The Prof had some idea about 

how the subject might be applicable to the Navy. 

He knew what he wanted to teach and the procedures by which he wanted 

to achieve his goal. He indicated he knew the good and bad points about 
the text and had additional material for the parts where it was needed. 


EVALUATION OF STUDENTS 


The student was well informed from the beginning exactly what was expected 
from him--and the instructor followed through by evaluating the student 
on those expectations. 


His exams reflected the material presented. His grading was fair and 
impartial. 


Tests, projects which gave an objective evaluation of my performance. 
This does not necessarily mean tests which were graded favorably. 


He was demanding in time and effort and his exams required a comprehensive 
knowledge. 
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Homework assignments were meaningful additions to his classroom presen- 
tation and the book, and not just busy work. 


Testing on what appears to be insignificant matters not covered by text 
or lecture in any degree of detail. 


Grading: either arbitrarily easy (i.e., much partial credit for obviously 
incorrect work, apparently in an attempt to camouflage poor instruction, ) 
or arbitrarily hard or inconsistent, and a supercurve. 


Examinations that were made to test the material as the student would 
be later using it vice emphasizing regurgitation. 


. Made many mistakes on the board while doing problems yet expected each 


answer to be exactly correct on exams. 


By appearing "hard'' all quarter he managed to completely kill interest 
in the subject matter and then award all his students with A's and B's 
~-not indicative of levels of learning at all. 


INSTRUCTOR ATTITUDE TOWARD STUDENT UNDERSTANDING 


The instructor displayed, and held, a genuine positive interest in the 
students’ progress in obtaining an understanding and knowledge of the 
subject material which he knew would be essential to the successful 
completion of follow-on ccurse. Through individual instruction, 
required problems, and frequent quizzes he monitored individual progress 


and was able to immediately spot any trouble spots. 


My belief that the instructor really cared whether or not the student 
understood the subject matter. He slowed down, repeated, answered the 
"stupid" questions until the student really did understand the subject. 
He gave the impression that he was happy to spend extra time with the 
student to insure the student learned. 


His sincere interest in imparting knowledge to his students vice just 
presenting material to meet the requirements of the curriculum. His 
goal for his students was understanding not memorization. 


A desire to see the student learn and understand material that is pre- 
sented, rather than to keep moving ahead just to remain "on schedule" 
and heck with understanding the material. 


He was interested only in ending the period and meeting requirements. 
He didn't really care if the students learned. 


» . . just "fulfilling his duty" by covering the topics instead of trying 
to teach us something. 


The real key to his success was that he was sincerely interested in the 
students understanding the material and made great efforts both inside 
and outside of class to insure this goal was achieved. 


~ 
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When I was able to determine that the professor's goal was to teach each 
and every individual in the class; via extra help sessions, office hours 
that were realistic, personal attitude, and receptivity. 


He realized his main function was instructing not research, writing 
papers, or writing a book. 


INSTRUCTOR ABILITY TO CREATE A LEARNING ENVIRONMENT AND DEMEANOR 


He always maintained a good humor, inserting personal examples of 
trying times he himself experienced so that the student was never 
made to feel stupid, ignorant, or thick. This rapport created a 
"learning atmosphere" highly conducive to education. 


The course was a required introductory course but completely unrelated 
to my major. My initial feelings were of the necessary evil, let's 

get it over with. About the middle of the quarter I discovered a keen 
interest in the subject had been kindled by the instructor and I wanted 
very much to pursue the subject. This kindling was done by the instruc- 
tor's use of personal experiences of about 20 years in the application 
of the discipline. He took abstract theory and made it live. 


He talked loudly enough so the whole class could hear. His board work 
was legible and clearly organized. He went slowly enough so that we 
had time to take notes and listen to the presentation. Handouts were 
used as an aid to understanding. 


The instructor's enthusiasm for the material being taught coupled with 
a genuine concern that the students were learning the material. 


Refusal to admit that he didn't know a point and failure to provide 
the information at a subsequent class session. - 


A course in which my major concentration seemed to focus on the dis- 
tracting mannerisms of the instructor. Rather than listening to what 
he said, I found myself counting the number of "ah's" and “it happens 
to be's" , . . unfortunately, these were not testable and my final 
mark was an indication of what I had concentrated on. 


His ability to take a dry, methodical subject and by giving sidelights 
as to the utility of the subject in wide areas, make the course an 


enjoyable experience. 


He was not constantly trying to prove how smart he was. 


INSTRUCTOR PREPARATION 


He was totally prepared for each lecture; i.e., examples were worked 

out in advance, homework problems were worked by the instructor prior 

to assignment (thereby avoiding lengthy exercises of questionable worth), 
student difficulties and questions were anticipated. 
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Trying to bullshit his way through class without having so much as read 
the assigned text. 


He simply didn't spend the necessary time to prepare his lectures in a 
manner that would enable the student to grasp the lecture material pre- 
sented. i 


Instructor totally prepared--pointed out problem areas before encountered. 
His continual confusion of basic principles due to lack of preparation. 


When the instructor spent 45 minutes of the classroom time working a 
homework problem and still didn't solve it. He was unprepared. 


Instructor assigned a problem for specific class period. On the day 
assigned, the instructor was not prepared to discuss the problem. He 
evaded direct questions and kept jumping to new material. The instructor 
asked for a consensus of opinion on the part of the students what "the 
most popular answer" was. 


The overwhelming emphasis on presenting material vice imparting knowledge. 
I am sure all my instructors possess a thorough knowledge in their 

fields. However, the majority lack in their ability to impart that 
knowledge. 


CLARITY OF EXPLANATION 


Outstanding lecture presentation that was clear, concise, and to the 
point. Most of all, his orderly manner of presentation from point A 
to point B and then a summary of high points during the last minutes 
of class. 

Most importantly, he presented the material in a manner that enabled 
me to understand a subject I had previously found very difficult. 


His willingness to offer complete explanation of classroom material 
when he could and his open admission that he couldn't when he couldn't. 


He could really get difficult concepts across to the class even if 
several different types of explanations were needed. He didn't leave 
anyone hanging. 


He was unclear, vague and often changed his position on statements. 
One never knew for sure if his class notes were correct or complete 
or even applicable. | 


His ability to draw on previous knowledge of the class to illustrate 
his points. 


He never finished explaining anything that he started. His tests 


appeared to be some of the things that he started to explain but 
never finished. 
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He has a way of saying things simply, and relating them to things I'm 
familiar with. 


~~ 


He could explain homework problems clearly. He illustrated high points 
with examples other than those read for homework. 


He acted self-satisfied, smug, and spouted a lot of jargon. He explained 
things I already knew, yet he managed to make them sound hopelessly 
complicated. 


INSTRUCTOR KNOWLEDGE 


The instructor not only knew the texts well, and could.quote directly 
and accurately from them, but he had just completed several years in 
the course environment. He had an opportunity to get a good compre- 
hensive view of his subject. HE WAS CURRENT. 


The feeling that the instructor knew what he was talking about and not 
merely reading out of the text. He could expand on what was in the 
text because of experience and competency. 


Intimate and unshakable knowledge of the area he was teaching. 


It was obvious that the teacher had no current exposure of real world 
problems, just the academic approach. 


INCOMPETENCE! e.g., a professor who can‘t do a problem on the board 
when the majority of the class can do it because they took the effort 
to read the pertinent section in the text. 


A feeling the instructor didn‘t know the material any better than I did. 


= 


The instructor lived in an academic vacuum with no idea how his course 
should or could relate to the world of a Naval Officer. 


A demonstrated inability in the area of instruction such as constant 
errors working sample problems. 


Instructor unsure of the material. He became extremely nervous when 
asked questions. His extemporaneous ramblings left me cold. 


Assigned work without having done it himself--consequently embarrassed 
when asked to do a problem and was not able to. 


INSTRUCTOR ABILITY TO TEACH AT APPROPRIATE LEVEL AND PACE 


He first established a common base of knowledge within the class. Even 
though he had to go back to a somewhat elementary level, the class was 
then with him all the way, and progress was very rapid and real. 
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Emphasis on getting the material covered vice teaching. 


The instructor did not move on to new material until he was satisfied 
everyone in the class understood. 


His ability to identify how well the class was understanding the material 
he was teaching. 


He would tell us over and over how important it was to keep moving 
whether we understood the previous material or not. 


The instructor lectured at a level consistent with his background and 
expertise, but right over the heads of many in the class. 


He vtelt compelled» to sticketo a daily routine, even if itemeant covering 
an extraordinary amount of difficult material in one or two days. 


He gloried in presenting abstract mathematical theories and persisted 
even though it should have been obvious that he had lost 80 - 9024 of 
the class. : 


The instructors did not draw on my previous knowledge, and taught 
wetow my level of intelligence. 


He spoke on my level, even though the material was over my head. 


The instructor had no feeling of how well the class was progressing 
and continuously glossed over information which the class is not 
understanding. 


Did not appear to be interested or aware that many students were 
failing to grasp important concepts of the course. 


INSTRUCTOR ATTITUDE TOWARD COURSE AND SUBJECT 
He failed to exhibit any enthusiasm in the basic course he was teaching 
and just "showed up" for each class. 
He was quite proud of his field, and, as a result, his students did 
not feel they were learning something irrelevant in the real life 
world. The Prof consistently related his topics to Navy-oriented 
problems. 
He was interested in his subject and projected this interest to me. 
I felt the instructor wasn't interested in what he was teaching. He 
seemed to have other things on his mind besides the class. He knew 


the material but couldn't teach. 


He acted like he was enjoying himself. He was full of his subject. 
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When the professor commenced the course with an apology for having to 
teach the course and for the students having to take the course. 


He was continually downgrading the importance of the subject. 


ORGANIZATION OF LECTURE MATERIAL 


He was organized. He explained things from beginning to end in a 
logical step-by-step process so that nothing was skipped and no doubts 
remained. 


The lectures were well organized and the presentation was held to the 
subject matter and the session not allowed to ramble or wander. 


The instructor obviously had it all together. I felt that he knew 
much more about the subject but was giving us the cream of what he 
knew in a well organized and clear fashion. (Not designed to snow 
students but to instruct them.) 


The instructor was disorganized and would often drift off the pertinent 
subject matter. So much so that with two weeks left in the course we 
had five weeks worth of important valuable material to cover. You can 
guess the result. 


Erratic, broken lectures resulting from inadequate organization. 


He presented the material in a manner such that the relevant material 
was stressed and nice to know material was not stressed so much. 


The instructor had a brilliant mind and a deep knowledge of the subject. 
But his presentations were so unorganized, his chalkboard work so jumpy 
from place to place, and his ability to orally communicate so limited 
(he would stop in mid-sentence to interject another thought and never 
return to the subject at hand) that he succeeded only in confusing 

the student. 


He continually went on unimportant tangents in lectures. 


INSTRUCTOR ATTITUDE TOWARD STUDENT 
The instructor treated me as an expert in my own field of Naval Affairs 
and related to me as an individual well into my own career. 


Instructor's attitude toward students, i.e., was respectful of their 
Opinions and questions. 


Making the student feel stupid: "That's so trivial I can't spend time 
Onetnat. | 
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Talking down to the students when most of the class had more practical 
experience in the matter than the professor. 


He spent time with students rather than research. He took a personal 
interest in each student in the class. 


Talking down to the class. 


INSTRUCTOR ATTITUDE TOWARD STUDENT QUESTIONS AND CONTENT 
OF STUDENT UNDERSTANDING 


He stressed important points which we could expect to see in higher 
level studies and was concerned with building good foundations. 


He fielded questions without making the student feel like an idiot 
for asking the question. 


Questions raised merely brought attacks on the questioner as to the 
student's inability to handle the particular problem. 


Never-mind-how-it~-works—just-plug-in-the-numbers type attitude. 
/ 


He became upset, even angry, if students questioned his methods. 
The instructor was concerned about concepts and understanding them 
rather than overwhelming students with insignificant dates and unrelated 


faets. 


His complete disregard for the questions that were asked and his failure 
to answer them. 


STUDENT COMPETENCE GAIN 


My most vivid recollection of instructor satisfaction was after the 
completion of a particular course. On reflection, I realized that the 
instructor had given us a conceptual approach to problem solving rather 
than a mechanical "plug and chug" approach that is common to the majority 
of such classes. 


I felt at the conclusion of the course that I had learned something 
and that it was useful. 


Satisfaction is coupled with the feeling that something is being 
obtained from the course. 


When, after six months, you can't remember anything about the course... 


We finished no smarter than when we started. 
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A feeling of competence in the subject after the course. 
Most of the information I gained from this course was on my own from 


reading and outside help from friends. I felt I could have missed 752 
of the class meetings and learned as much. 


INSTRUCTOR ABILITY TO CONTROL CLASS TIME 


The effectiveness he demonstrated in encouraging class participation. 


He did not try to cram 80 minutes of lecture in a 50 minute time frame. 
He left time to clear up on-the-spot questions. 


Wasted no time. If he didn't have any more lecture material, he stopped 
lecturing and class was over. 


INSTRUCTOR AVAILABILITY 


He was never too busy to talk to a student and did not give the appear- 
ance of being bothered or held up--in fact, he seemed to like to talk 
to you if you went to see him with a question. 


He was available for consultation at almost anytime outside class, 
rather than only during office hours, 


He was available at any time to go over difficult areas with students. 


If it was good golf weather his door was locked. If the weather was 
poor, it was closed. He kept us informed of his scores. 


INSTRUCTOR ABILITY TO EVOKE TNTEREST 


The instructor assigned readings from the required text and then every 
lecture hour repeated, often verbatim, the material from the text. It 
is difficult to listen to an instructor re-read to you for an hour 

the material you read the night before. 


He made his presentations interesting, if not downright entertaining, 
by keeping them relevant, by using real world situations as examples, 
and by a highly developed sense of humor. 


Attempt on part of the instructor to make even the most unappealing 
part of the course interesting. 


Taught from lesson plans and book. Instructor knew equation and figure 
numbers "by heart"... he had taught the course so many times it had 


very little appeal. 


He used interesting examples to illustrate his topic. 
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